The Art of Data Visualization: Detecting Multivariate Data Outliers Using an Interactive Approach

Successfully detecting outliers in multivariate data requires statistical and programming skills and can be very time consuming. Requests for outlier detection can come from different skills groups therefore it is more efficient and effective to allow users to interact directly with the data themselves. We have developed an interactive, web based data visualization application for outlier detection using R Shiny that does not require specialist knowledge to use. This application reads in various file types, manipulates and reduces datasets as needed, performs an array of different outlier detection methods, visually displays the outlier output through interactive graphs and downloadable tables, and provides tests to check any distributional assumptions. This paper visually demonstrates the functionality of the application, which includes exporting all or a subset of the outliers displayed and utilizing machine learning techniques to better predict outliers based on prior decisions made by the user. INTRODUCTION An outlier is an observation that appears to deviate markedly from other observations in the sample. Outlier detection is necessary for two key reasons: firstly, for data cleaning, so that potentially corrupt or inaccurate records can be identified and secondly, to ensure that extreme values are appropriately addressed so as not to skew the analyses. It can be a tedious task deciding which outlier detection method to use and reviewing the validity of tests through the checking of distributional assumptions. It also traditionally requires a programmer or statistician to execute the decided technique and produce any supportive visuals, which can be problematic for less technical professionals or those uncertain as to the options/methods available. Detecting outliers is necessary but can be time and labor intensive. We have streamlined this process with an interactive tool that leads to earlier detection of outliers and hence more time left to focus on the primary analysis. Our outlier application speeds up outlier detection tasks by utilizing flexible data file types, accompanied by the ability to manipulate the dataset as necessary and apply an array of outlier techniques for ease of the user. It also checks any distributional assumptions, and provides different techniques based on the dimensionality of the data. All of the techniques provide supporting visuals before and after the outliers have been removed to further assist in the decision making.